Sparse single-index model

Let (X, Y ) be a random pair taking values in Rp × R. In the socalled single-index model, one has Y = f?(θ?TX) + W , where f? is an unknown univariate measurable function, θ? is an unknown vector in Rd, and W denotes a random noise satisfying E[W |X] = 0. The single-index model is known to offer a flexible way to model a variety of high-dimensional real-world phenomena. However, despite its relative simplicity, this dimension reduction scheme is faced with severe complications as soon as the underlying dimension becomes larger than the number of observations (“p larger than n” paradigm). To circumvent this difficulty, we consider the single-index model estimation problem from a sparsity perspective using a PAC-Bayesian Corresponding author. Research partially supported by the French National Research Agency under grant ANR-09-BLAN-0128 “PARCIMONIE”. Research partially supported by the French National Research Agency under grant ANR-09-BLAN-0051-02 “CLARA”. Research carried out within the INRIA project “CLASSIC” hosted by Ecole Normale Supérieure and CNRS.